1.1. Field of the Invention
The invention concerns systems for automatically analyzing echocardiographic digital images of the heart, especially two-dimensional images acquired using apical four-chamber view of the heart. The systems preferably employ a collection of matched filters on the images to automatically locate and measure features of the heart.
1.2. Description of Related Art
Two-dimensional ultrasonic imaging is used as an important non-invasive technique in the comprehensive characterization of a number of body organs. In ultrasonic imaging, a sound pulse is sent along a ray from a transducer towards the organ that is being imaged. The pulse is attenuated and reflected when it hits a medium with an acoustic impedance different from that of the medium in which the pulse is traveling. The time the sound pulse takes in transit is a measure of the distance of the medium interface from the transducer. The amount of energy that is reflected is a measure of the difference in acoustic impedance across the interface. Since the energy of the pulse diminishes as it travels, post-processing of the reflected signal includes time/gain control that compensates for the attenuation of the signal over time. Assuming the pulse travels at a single speed in the body, and by taking rays uniformly distributed across a given plane, a two-dimensional record of the received energy in spatial (Cartesian, polar) coordinates can be used to present a cross-sectional view of the imaged organ.
Echocardiography is the application of ultrasonic imaging to the heart. Echocardiography has experienced widespread acceptance in the evaluation of cardiac disease, structure, and function of the heart. This acceptance is in large part due to its non-invasive nature, and to its real-time capability for observing both cardiac structure and motion. Using echocardiography, quantitative information may be obtained concerning cardiac anatomy, chamber diameter and volume, wall thickness, valvular structure, ejection fraction, etc. (Weyman, 1994).
The real-time capability of echocardiography may be used to measure variations in the shape of heart structures throughout the cardiac cycle (Weyman et al., 1984). These analyses require the complete determination of inner (endocardial) and outer (epicardial) boundaries of the heart wall, particularly of the left ventricle. Present evidence indicates that sensitive detection of ischemic disease with two-dimensional echocardiography requires knowledge of the endocardial border on echocardiographic frames throughout the cardiac cycle (Weyman et al., 1984).
Because both global and regional left ventricular function are major variables used to determine prognosis in cardiac disease, there is considerable interest in the ability to quantify function indexes from echocardiographic images. Presently, such indexes, e.g., left ventricular chamber volume and left ventricular ejection fraction, are calculated from observer-defined cardiac boundaries traced with either a light pen, a digitizing tablet, or either a mouse, trackball, or other suitable computer input device. Tracing of endocardial borders on two-dimensional echocardiograms, however, is tedious and the selected borders may be highly subjective. Indeed, in most systematic studies, substantial intra-observer and inter-observer variability has been found in such observer-defined cardiac boundaries (Weyman et al., 1984). An echocardiogram is a generic term for an image formed using ultrasound. It contrasts with images produced by x-ray, magnetic resonance, or other techniques (which are typically referred to as xe2x80x9cradiologic medical tomographic imagesxe2x80x9d). An echocardiogram may also be referred to as a xe2x80x9csonogram,xe2x80x9d and is an image that is formed using ultrasound as the type of wave producing the images.
One advantage of ultrasound (sonograms) as an image formation technique, is that images can be formed very rapidly, meaning typically 10 to possibly 80 times per second, therefore, since the heart has one beat per cycle, it is feasible to catch up to typically 30 to 50 images in this series through one heart cycle. The registration point moves, or at least can move on every frame and therefore, in both the short axis and the long axis, the inventors have elected to locate that point on each frame. It can not be assumed to stay fixed. Measurements may be made on each frame (i.e. a single image from a series of images) and adjusted to the size of the heart by taking into account the scale factor of the image; in other words, how many pixels represent a centimeter can be changed by the operator. The scale factor relating the number of pixels in one centimeter in the image must be known so that the measurements can be adjusted to the true size of the heart. The xe2x80x9cregistration pointxe2x80x9d is the origin of the system of coordinates.
Manually defining such boundaries becomes increasingly labor intensive when the analysis of a complete cardiac cycle is needed to provide a description of the systolic and diastolic wall motion pattern, or when a number of echocardiographic frames have to be processed in order to obtain an extended time-history of cardiac function. It is therefore desirable to automate as much as possible the determination of boundaries of echocardiographic images, as well as other structural features. Automated definition of the boundaries and features would improve the reliability of analyses by eliminating the subjectivity of manual tracing.
In the past several years, advances in computer data processing technology have allowed the application of several different automatic boundary detection methods to echocardiographic images (Conetta et al., 1985). However, most researchers have had difficulties with image enhancement and boundary detection with echocardiographic images because of the low signal-to-noise ratio and large discontinuities in such images. Thus, automated border detection has been reported in two-dimensional echocardiographic images, but only when the images have been of good quality and certain smoothing techniques have been employed prior to edge detection in order to render the endocardial edge more continuous. An overview of the field is set forth in Kerber (1988).
U.S. Pat. No. 5,797,396 to Geiser and Wilson (1998) (specifically incorporated herein by reference in its entirety) describes a method for the automated analysis of short-axis views of a heart.
1.3 Deficiencies in the Prior Art
What is lacking in the prior art is a method for automatically determining quantitative characteristics of ultrasonic images, especially long-axis echocardiographic images that provide details of the apical four-chamber view of a heart. In particular, there is also a need for a method that can automatically determine key regions of an imaged structure and approximate the borders of such a structure.
The present invention is directed at making automated measurements on echocardiographic 2-D images acquired using the apical 4-chamber view. In a broad aspect, the invention comprises a system for processing 2-D digital images of the heart to analyze the structure and functioning of the heart. These features include the right and left ventricles, the septum, the mitral valve apparatus and the myocardium.
The invention provides a method for automatically analyzing a long-axis image of a heart, the method comprising the steps of: (a) generating an image frame of the myocardium, interventricular septum, and mitral valve annulus of the heart, the image frame comprising a plurality of rows and a plurality of columns of pixels in a digital format; (b) determining an approximate position of the interventricular septum from the diagnostic image comprising: (i) passing a filter through the image to determine a maximum mean pixel intensity, wherein the maximum is a first approximate position of the interventricular septum; (ii) defining a second approximate position of the interventricular septum with a series of straight line filters; (iii) obtaining a best fit line through the second approximate position from the upper portion of the series of straight line filters; (c) determining an approximate position of a medial border of the mitral valve annulus by passing a right-angle filter down the best fit line determined from step (b)(iii) until a maximum mean pixel intensity is determined, wherein the maximum is indicative of the approximate position of the mitral valve annulus; (d) defining a registration point as the point at which the best fit line intersects the approximate position of the mitral valve annulus; (e) locating one or more regions of the myocardium in relation to the registration point; and (f) calculating the mean pixel intensity in each of the regions of the myocardium, wherein the mean pixel intensity provides an analysis of the long axis image of the heart.
In certain preferred aspects of the present invention, the image is generated by ultrasound, X-rays, such as planar X-rays or tomographic X-rays, or magnetic resonance imaging. In a particularly preferred embodiment of the invention, the image is generated by ultrasound.
In particular embodiments, the filter is a large mean, first derivative, or second derivative filter. In other aspects of the invention, the series of straight line filters comprises one or more Laplacian or first derivative filters, and in preferred embodiments, the series of straight line filters comprises about 32 Laplacian filters.
In certain aspects of the invention, the regions of the myocardium are selected manually for densitometry over a coronary perfusion bed, while in other aspects, the regions of the myocardium are selected automatically for densitometry.
The invention also provides a method for determining the optical density of a digital image of an apical view of a heart, the method comprising the steps of: (a) obtaining a control frame from a baseline period prior to injecting a contrast agent; (b) determining a first registration point in the image; (c) locating one or more regions of a myocardium in relation to the registration point on the control frame; (d) calculating the mean pixel intensity of the one or more regions of the myocardium; (e) injecting the contrast agent and obtaining a sequence of frames; (f) determining a second registration point in one or more of the frames of the sequence; (g) locating one or more regions of the myocardium on one or more of the frames corresponding to the regions of a control frame; (h) calculating the mean pixel intensity of the one or more regions of the myocardium in one or more of the frames of the sequence; and (i) obtaining a time-mean pixel intensity curve by comparing the mean pixel intensity for the frames following injection of the contrast agent to the control frame, wherein the time-mean pixel intensity curve determines the optical density of the digital image in a selected region of the apical long axis view of the heart.
In certain aspects of the invention, the method further comprises the step of determining whether perfusion is present in a region by comparing a pre-contrast mean pixel intensity to that calculated after peripheral injection. Preferred contrast agents for use in the present invention include, but are not limited to, Albunex(copyright) or Optison(copyright).
The present invention further provides a method for automatically locating an approximate position of the interventricular septum of a heart in a long-axis image, the method comprising the steps of generating an image frame of the heart, the image frame comprising a plurality of rows and a plurality of columns of pixels in a digital format, and passing a filter through the image to determine a maximum mean pixel intensity, wherein the maximum is an approximate position of the interventricular septum in the image. In particular aspects, the method further comprises passing a series of straight line filters through the image.
The invention also provides a method of automatically locating an approximate position of a medial border of a mitral valve annulus in a long-axis diagnostic image of a heart, the method comprising the steps of: (a) generating an image frame of the interventricular septum and mitral valve annulus of the heart, the image frame comprising a plurality of rows and a plurality of pixels in a digital format; (b) determining an approximate position of the septum from the diagnostic image comprising: (i) passing a filter through the image to determine a maximum mean pixel intensity, wherein the maximum is a first approximate position of the interventricular septum; (ii) defining a second approximate position of the interventricular septum with a series of straight line filters; (iii) obtaining a best fit line through the second approximate positions from the upper portion of the series of straight line filters; and (c) determining an approximate position of a medial border of the mitral valve annulus by passing a right-angle filter down the best fit line determined from step (b)(iii) until a maximum mean pixel intensity is determined, wherein the maximum is indicative of the approximate position of the mitral valve annulus.
Additionally, the present invention provides a method of screening a compound for use as a contrast agent in echocardiography, the method comprising the steps of: (a) obtaining, from a baseline period prior to injecting the compound, a control frame of a digital image of an apical long-axis view of a heart; (b) determining a first registration point in the image; (c) locating one or more regions of a myocardium in relation to the registration point on the control frame; (d) calculating the mean pixel intensity of the one or more regions of the myocardium; (e) injecting the compound and obtaining a sequence of frames; (f) determining a second registration point in the frames; (g) locating one or more regions of the myocardium on the frame corresponding to the regions of the control frame; (h) calculating the mean pixel intensity of the one or more regions of the myocardium in the frame; and (i) obtaining a time-mean pixel intensity curve by comparing the mean pixel intensity for the frame following injection of the contrast agent to the control frame, wherein an increase in the mean pixel intensity in the regions of the myocardium in the frame following injection of the contrast agent over the control frame is indicative of injection of a contrast agent.
In a most general sense, the algorithm concerns: (a) identification of the sector scan; (b) finding vertical line through the middle of the LV chamber (i.e. through the low intensity (or dark) pixels on the right side of the image). This method uses all of the frames in the sequence between ED and ES. The purpose of these steps is to limit the region of computation for the septum. If this step is omitted, the septum is sometimes found too far to the right on the free wall of the RV; (c) finding the location and computing the slope of the septum; (d) locate the left side of the mitral valve ring (i.e. the medial MV) using a matched filter designed using a meaned first derivative in the vertical direction; (e) locate the right side of the mitral valve ring (i.e. the medial Mv) using a matched filter designed using a meaned first derivative in the vertical direction; (f) compute the location of the apex using a matched filter designed form a meaned first derivative; (g) compute border points of the septum; and (h) compute border points of the free wall (i.e. right side of LV).
The most important boundaries sought are the boundaries of the myocardium, that is the endocardium or boundary of the inner lining of the heart and the epicardium, which, in this case, is considered the lining of the right side of the septum and true epicardium on the apex, inferior, and lateral walls. These are shown, and identified in FIG. 1.
An important piece of information that may be extracted from an apical views of a heart is the length of the left ventricle; in other words, the distance from the mitral valve plane to the apex, as well as the diameters. This allow a calculation of volume and estimation of ejection fraction (or fractional ejection of blood volume over time) which is a measure of the global function of the heart. In addition, if the entire inner lining of the heart(endocardium) is identified by this method then motion of individual portions of the wall over time can be mapped. This allows the particular features related to abnormal motions, such as with a heart attack, to be located and measured for the percentage of heart muscle involved.
In one broad method embodiment of the invention, a 2-D, long axis, ultrasonic image of the heart is filtered to identify selected featuresxe2x80x94notably the septum and the medial border of the mitral valve annulus. These two features intersect at right angles and provide a useful point for locating and orienting these and other features and functions of the heart. Thus, this registration point serves as the origin of a system of coordinates for defining the location of pixels and structural features in the image of the heart. The image typically comprises rows and columns of pixels in a digital format with the registration point as a reference point for the rows and columns.
Filtering and digital imaging is performed by passing what is referred to as a xe2x80x9ckernelxe2x80x9d over every pixel in the image to obtain a resulting image, as described in U.S. Pat. No. 5,797,396. While all filters will have a maximum and minimum value resulting from the kernel, what may be important at the time may be the maximum value, the minimum value, or even which pixels are zero after the filter is passed through the image. The filters provide a digital number as the output, the person designing the algorithm decides whether it is important to seek the maximum, the minimum, the zero crossings, or some other level of output from the filters. The inventors believe that a more generic term for filters would be convolutions. In general, all of the filters in the disclosed processes are referred to as adaptive, meaning that some features of the image as it is calculated actually determine the size and shape of the filters in the next step.
The xe2x80x9clong axisxe2x80x9d refers to an image of the heart which visualizes structures in a plane parallel to the longitudinal axis; in other words, the axis extending from the apex or tip of the heart to the base, which can include the aorta and posterior portions of the left and right atrium. While a parastemal long axis view can be obtained (that is a view from along side the sternum) a preferred long axis view in the present invention is the apical long-axis view; a view which is taken typically from the 5th or 6th intercostal space near the anterior axillary line and is taken from that position on the lateral chest aiming back towards the base of the heart. The typical orientation is then one where the apex of the heart is near the apex of the sector scan with the left ventricle to the left, right ventricle to the right and then posterior to these, near the bottom of the scan, the left atrium again in the left posterior portion and the right atrium in the fight posterior portion of the scan. In a preferred long axis view, the long axis passes through the epicardial apex. It is realized by echocardiographers, however, the probe or ultrasound transducer can frequently not be placed in a position to achieve this. What is expected as the best possible position which maximizes the long axis length of the left ventricle; that is, the distance from the epicardial apex to the mitral valve plane.
Typically such a view determined, by placing the transducer on the chest wall in an intercostal space closest to the apex of the heart and then aimed in a direction towards the mid base of the heart, meaning the mid mitral and tricuspid valve rings and oriented in such an angulation as to maximize the size of the left and fight ventricle and left and fight atrium. Such a view contrasts markedly to the apical 2 chamber view, which is a view perpendicular to the view described in the present invention. In the apical two-chamber view (described in inter alia U.S. Pat. No. 5,797,396), the transducer is aimed through the maximum short axis diameter of the left ventricle and left atrium only.
Usually for imaging in this view, the patient is in a left lateral decubitus position (lying on their left side), so that the apex of the heart is positioned hanging near the left chest wall. As mentioned above, the long axis view does not necessarily always have to be from the apical position, but that is the preferred view for practice of the present invention.
A variety of filters may be employed to process images in accordance with the invention. In a first step it is preferred to pass one or more filters laterally across a long axis image or parallel to the x-axis to locate the position of the septum. Following this, a right angle filter is passed vertically or parallel relative to the y-axis to locate the lateral border of the mitral valve annulus. The intersection of the septum and the medial border of the mitral valve annulus may then serve as the registration point of a coordinate system.
It has been found that location of the registration point is best obtained by employing a sequence of different filters. Thus, in locating the septum, it is preferred in a first filtration step to pass a mean filter across a long axis image to determine a maximum mean pixel intensity (brightness). The location of the maximum mean pixel intensity reveals a first approximate location of the septum. Since the septum may not be completely vertical in the images, a series of straight line filters, each comprising a plurality of rows, is then passed across the image to identify the first approximate location of the septum. The straight line filters are preferably Laplacian filters, arranged one above the other in abutting relation. These straight line filters are used to find the centerline of the left ventricle (LV) (shown in steps 2-6). The location of the maximum mean pixel intensity for each straight line filter is determined to thereby provide a second approximate location of the septum in each of the areas of the image scanned by the straight line filters. By creating a least-squares best fit line through the second approximate location of the septum, the slope of the septum in the image may be better defined. Empirical studies have shown that the upper portion of the second approximate location of the septum is preferred for determining the best fit line, and that the upper two-thirds of the second approximate location of the septum is most preferred. A xe2x80x9cbest fitxe2x80x9d line is usually arrived at by a correlation procedure or a least squares operation and is a standard mathematical technique.
If it proves desirable to further define the location of the registration point in an image, this may be done by locating additional structures of the heart in the image. For example, after locating the point of intersection of the best fit line representing the septum and the location of the medial border of the mitral valve annulus, one may rotate (how) the right-angle filter around the axis of their intersection point until a maximum mean pixel intensity is determined. This maximum provides an approximate location of the lateral border of the mitral valve annulus. The intersection point of the lateral and medial borders of the mitral valve annulus and the best fit line representing the septum may then be used as the registration point. In some instances this point may be more precisely defined when comparing multiple image frames than the registration point determined by just using the intersection point of the medial border and the best fit line representing the septum.
As indicated above, when additional specific regions of the myocardium are defined, their location in the image in relation to the registration point may be determined. In preferred embodiments, this may be achieved by determining the coordinates (x, y) of each pixel in the defined regions in relation to the registration point. Then, when comparing a series of image frames, to find the same region of the myocardium in any specific image frame, the registration point is determined in the specific frame and the regions are automatically redrawn based on the coordinates established in the first frame.
It will be recognized that if the various regions of the myocardium were to be defined by a specific set of coordinates, movement of the heart due to breathing or other causation may cause different areas of the myocardium to fall within a given region in a particular frame or that given region may no longer contain any portion of the myocardium. The present invention, however, by defining a particular registration point in terms of heart structure also defines the locations of the regions of the heart in relation to that point and compensation may be made for heart movement. This is particularly true, for example, when frames taken from a specific junction of the heart cycle (e.g., diastole) are compared because the heart tends to have the same shape at the same point in its cycle.
The present invention may have particular utility in determining changes in pixel intensity of a digital image of an apical, long axis view of the heart during application of certain contrast agents. This may be done by obtaining a control frame from a baseline period or interval of the heart cycle prior to injecting a contrasting agent. A registration point is first determined in the control frame as described above. One or more regions of the myocardium are then located in the control frame image in relation to the registration point, and the mean pixel (intensities) of the regions of the myocardium on the control frame determined. The contrast agent may then be injected. Suitable contrasting agents include Albunex(copyright) and perfluorocarbon compounds such as Optison(copyright). A preferred agent is Optison(copyright), which is manufactured by Molecular Biosystems, Inc., and available from Mallinckrodt, Inc. (St. Louis, Mo.).
After injection of the contrasting agent, a sequence of end-diastole (ED) frames are obtained. In each of these succeeding frames a registration point in the frame is obtained as described above, thereby locating one or more regions of the myocardium on the ED frame corresponding to the regions on the control frame where the mean pixel (intensity) of the region is calculated. By comparing the mean pixel (intensities) for the ED frame following injection of the contrast agent to those of the control frame, a time-mean brightness curve may be obtained. The time-mean brightness curve for a region of the myocardium may be used to determine whether perfusion is present in the region.
In an overall and general sense, the method provides means for identifying the position of principal major structures in an echocardiographic image of a heart and using the position of these structures to guide a search for finer and finer detailed structures in the image. In other words, the inventors have searched for the position of the interventricular septum first (FIG. 2) as the gross first step and then the slope of the septum as the second step (FIG. 3) using this, the next step is to find the position of the mitral valve plane which is the next gross structure in the image and this is performed using the right angle filter (FIG. 5). This arrives at the reference point noted in FIG. 6, which subsequently allows search for the lateral mitral valve annulus. Knowing the reference point and position of the lateral mitral valve annulus, the midpoint of this line represents the center of the mitral valve plane and is the approximate position from which to measure the long axis length of the ventricle, which extends from this point to the apex epicardium. Alternatively, one might search for the mitral valve plane first and then use that position to search for the septum, which method represents another useful aspect of the invention. A third approach may be to search for the largest blood pool or in other words, use the large black area at the center of the left ventricle as the principle feature in the image. The inventors contemplate, that in its most essential rudimentary form, the invention is using whatever large feature one considers most prominent and then working in a logical fashion from that large feature to the finer detailed positions in the image using adaptive step wise implementation of filters and thresholds to finally locate key positions, such as the positions of the mitral valve annulus (lateral and medial), aortic annulus position, apex of the left ventricle, etc. And finally, achieving the best location for the endocardium and epicardium of the left ventricle for the reasons described above.